Motion compensated video processing

ABSTRACT

A motion compensated video processing method includes encoding a video frame group I 0  -I 7  into one in which one frame I 0  of the group is a basic input frame or `I `frame, and the other frames B 1  -B 7  are derived from calculating the difference between a current frame and two neighbouring frames so as to form `B ` frames. The `B ` frames are derived in a logarithmic progression. The `B ` frames are quantized by applying selective quantization weighting to predetermined input frames. Encoding can be of the forms described as open loop or closed loop, preferably open loop. Decoding can be achieved by applying inverse quantization weighting to the input frames prior to selective summnation in order to achieve a constant reconstruction error for each frame even though quantization errors carry over from each reconstruction stage.

This application is a division of application Ser. No. 08/658.592 ,filed Jun. 5, 1996.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to motion compensated video processing methodsand apparatus.

2. Description of the Prior Art

It is known to use motion compensated predictive coding in videoprocessing so as to compress the data required for coding. One suchdigital compression standard, known as MPEG1, was set by the MotionPicture Experts Group (MPEG) of the International Standards Organisation(ISO). A further developed standard is known as MPEG2. This standardmakes use of (i) forward prediction from past to current frames of avideo signal, and (ii) bidirectional prediction from past to currentframes, and from future to current frames.

Standard MPEG coding uses a convention of designating frames as either`I`, `B ` or `P ` frames. An `I ` frame is a picture coded as a singleframe, whereas a `P` frame is the output of a predicted framedifference, and a `B ` frame is the difference between a current frameand the average of two frames either side (Bi-directional). Theconvention is now well understood and documented, for example in ISO/IEC11172-2:1993(E) "Information Technology--Coding of Moving Pictures andAssociated Audio for Digital Storage Media at up to about 1.5 Mbit/s".

Early forms of MPEGI used only forward prediction using an initial `I`frame followed by a number of `P ` frames as follows:

I P P P . . . P I

Later versions of MPEG1 as well as MPEG2 use all three types, typicallyin a 12 frame sequence (for 50 Hz applications) as follows:

I B B P B B P B B P B B I

In each method, motion vectors are applied to minimize the predictionerror and improve coding efficiency.

Although the MPEG standards provide an effective degree of compression,there is a need for systems and methods allowing still greatercompression in video processing.

SUMMARY OF THE INVENTION

It is an object of the invention to provide a motion compensated videoprocessing method which allows a greater degree of compression thanpreviously-proposed methods.

It is another object of the invention to provide a motion compensatedvideo processing method which allows a simple open-loop encodingtechnique to be utilized.

According to one aspect of the invention there is provided a motioncompensated video processing method comprising encoding a video framegroup into one of 2_(n) frames in which one frame of the encoded groupis a basic input frame and the or each other frame is derived fromcalculating the difference between a current frame and two neighbouringframes on either side.

According to another aspect of the invention there is provided a motioncompensated video processing method comprising encoding a video framegroup into one having an `I ` frame, the or each other frame being a `B` frame.

A preferred embodiment of the invention utilizes a temporal decimationmethod to effect video compression in which a sequence or group of videoframes, otherwise known as a group of pictures (GOP), such as 2, 4, 8,16, 32 (etc.) frames, is decomposed into one still frame and theremaining frames as difference frames. The method is referred to aslogarithmic temporal decimation. The method uses only `I` and `B `frames, in which an `I ` frame can be considered to be a normal frame,and a `B ` frame can be considered to be the result of calculating thedifference between a current frame and two neighbouring frames on eitherside. Motion vectors are normally used to assist the creation of `B `frames. The method creates only `B` frames as the result of applying aprogressive structure in both encoding and decoding. The `B ` frames arequantized by applying selective quantization weighting to predeterminedinput frames. Decoding can be achieved by applying inverse quantizationweighting to the input frames prior to selective summation in order toachieve a constant reconstruction error for each frame even thoughquantization errors carry over from each reconstruction stage. Themethod achieves better results than either MPEG1 or MPEG2 coding.Further, use of this form of compression enables the use of a simpleropen-loop encoding technique giving better visual results.

The above, and other objects, features and advantages of this inventionwill be apparent from the following detailed description of illustrativeembodiments which is to be read in connection with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic of a logarithmic temporal decimation structureaccording to an embodiment of the invention;

FIG. 2 shows a logarithmic temporal interpolation structure which can beused to decode the frame sequence provided in FIG. 1;

FIG. 3 shows a basic temporal coding process of closed-loop type;

FIG. 4 is a pictorial representation of noise components of closed-loopand open-loop encoding systems;

FIG. 5 is an encoder timing diagram showing the timing on aframe-by-frame basis;

FIG. 6a is a decoder timing diagram for a known type of decoder; and

FIG. 6b is a decoder timing diagram for a reduced timing delay decoder.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The `B ` frame octave structure embodying the invention will now bedescribed.

The number of frames between the `I ` frames is a power of 2; i.e. 2, 4,8, 16 etc. A 2-frame sequence is the trivial case as follows:

I₀ B₁ I₂ B₃ I₄ B₅ I₆ B₇ I₈ etc.

Each `B ` frame is predicted from the `I ` frames either side, with orwithout motion vector compensation. This is an efficient codingtechnique, but the number of `I ` frames is half the total frame count,thereby limiting coding efficiency.

The next step is to replace every other `I ` frame (I₂ and I₆) by acorresponding predicted `B ` frame from above as follows:

I₀ B₁ B₂ B₃ I₄ B₅ B₆ B₇ I₈ etc.

Frame B₂ has been created by the difference: I₂ -(I₀ +I₄)/2,

and frame B₆ from the difference: I₆ -(I₄ +I₈)/2.

This gives a 4-frame sequence in two stages. The efficiency is quitegood, but there is further advantage to be gained in coding I₄ into a `B` frame using a prediction from frames I₀ and I₈ as follows:

B₄ =I₄ -(I₀ +I₈)/2

This leads to the sequence:

I₀ B₁ B₂ B₃ B₄ B₅ B₆ B₇ I₈ etc.

The process can be continued indefinitely with the `I ` frames beingspaced by twice the temporal distance at each stage. However, the codingefficiency rapidly approaches an asymptotic limit which is almostreached with a 3-level decimation and further decimation is not normallynecessary.

It is also the case that coding delays increase with higher levels ofdecimation and such delays may introduce operational problems. Thememory requirements also increase with higher decimation levels so theincentive is to use as few levels of temporal decimation as possiblecommensurate with desired picture quality. In most experiments, even forthe highest compression ratios, 3 stages of temporal decimation haveproven sufficient.

FIG. 1 shows the 3-level, 8-frame logarithmic decimation process indiagrammatic form. Note that although shown as a simple arithmeticprocess, at each stage, new `B ` frames are created from surrounding `I` frames with the addition of motion vector compensation in order tominimize the `B ` frame errors.

The motion vectors are related to adjacent frames for the generation ofB₁, B₃, B₅ and B₇. For the generation of frames B₂ and B₆ the motionvectors are generated from 2-frames distant. And for the generation offrame B₄, the motion vectors are generated from 4-frames distant (i.e.,from frame I₀ and I₈,). This means that the motion vector range mustincrease by a power of 2 for each stage in the logarithmic decimationprocess.

The decoding process is shown in FIG. 2.

For the 8-frame sequence shown as an example, the decoder mustregenerate the frames in a particular sequence for the following reason.Take, for example, frame 1. Its reconstruction depends on having frames10 and 12. But frame 12 must be reconstructed from frames I₀ and I₄ ;and I₄ must be reconstructed from frames I₀ and I₈. The decoder must,therefore, reconstruct the frames in a particular order as follows:

    ______________________________________                                        From frames I.sub.0 and I.sub.8 :                                                                I.sub.4 = B.sub.4 + (I.sub.0 + I.sub.8)/2                  From frames I.sub.0, I.sub.4 and I.sub.8 :                                                       I.sub.2 = B.sub.2 + (I.sub.0 + I.sub.4)/2                                     I.sub.6 = B.sub.6 + (I.sub.4 + I.sub.8)/2                  From frames I.sub.0, I.sub.2, I.sub.4, I.sub.6 and I.sub.8 :                                     I.sub.1 = B.sub.1 + (I.sub.0 + I.sub.2)/2                                     I.sub.3 = B.sub.1 + (I.sub.2 + I.sub.4)/2                                     I.sub.5 = B.sub.1 + (I.sub.4 + I.sub.6)/2                                     I.sub.7 = B.sub.1 + (I.sub.6 + I.sub.8)/2                  ______________________________________                                    

Although the decoder must reconstruct the frames in a particular order,the general encoder could encode in any sequence except in the followingcircumstance in which the aim is to minimize error propagation duringencoding.

The classic DPCM coding process encompasses a decoder within the encoderforming a loop in order to prevent recursive errors. The same problemoccurs in both `P `and `B `differential frame coders, so MPEG arrangesto overcome this by the loop shown in FIG. 3 which illustrates a basictemporal coding process of closed loop type.

Referring to FIG. 3, the I frames F₀ and F₂ are processed first,resulting in reproduced frames F₀ +ε₀ and F₂ +ε₂ respectively. Theseframes have errors introduced by the quantization process. The averageof the two frames is subtracted from frame F₁ to produce the differenceframe B₁. This is coded thus producing its own error to be added to theoriginal signal.

Noting that the receiver also produces frames F₀ +ε₀ and F₂ +ε₂, thenthe reconstruction of frame F₁ cancels the error propagation leaving F₁+ε₁. Thus, although the encoder is more complex, the errors introducedby each frame are local to that frame. However, it should also be notedthat the noise introduced by the prediction from the reconstructedframes, F₀ +ε₀ and F₂ +ε₂, introduces an imperfect prediction matchwhich can worsen the `B ` frame prediction.

Consider, now, the problem which would occur if such feedback were notused in an I-P based system such as MPEG1. The `I ` frame is coded asnormal, The `P ` frames are created from the original pictures withoutreconstruction errors.

In the transmitter, the encoded frames are as follows:

    ______________________________________                                        Start Frame   Encoded Frame                                                                            Decoded Frame                                        ______________________________________                                        I.sub.0       I.sub.0    I.sub.0 + ε.sub.0                            I.sub.1       P.sub.1 = I.sub.1 - I.sub.0                                                              P.sub.1 + ε.sub.1                            I.sub.2       P.sub.2 = I.sub.2 - I.sub.1                                                              P.sub.2 + ε.sub.2                            I.sub.3       P.sub.3 = I.sub.3 - I.sub.2                                                              P.sub.3 + ε.sub.3                            I.sub.4       P.sub.4 = I.sub.4 - I.sub.3                                                              P.sub.4 + ε.sub.4                            . . . etc.                                                                    ______________________________________                                    

The receiver recreates frames as follows:

    __________________________________________________________________________    Input                                                                             Decoded Frame (a)      Decoded Frame (b)                                  __________________________________________________________________________    I.sub.0 + ε.sub.0                                                         I.sub.0 + ε.sub.0                                                                            I.sub.0 + ε.sub.0                          P.sub.1 + ε.sub.1                                                         I.sub.1 ' = P.sub.1 + ε.sub.1 + I.sub.0 + ε.sub.0                                    I.sub.1 + ε.sub.1 + ε.sub.0        P.sub.2 + ε.sub.2                                                         I.sub.2 ' = P.sub.2 + ε.sub.2 + P.sub.1 + ε.sub.1 +           I.sub.0 + ε.sub.0                                                                            I.sub.2 + ε.sub.2 + ε.sub.1 +                                 ε.sub.0                                    P.sub.3 + ε.sub.3                                                         I.sub.3 ' = P.sub.3 + ε.sub.3 + P.sub.2 + ε.sub.2 +           P.sub.1 + ε.sub.1 + I.sub.0 + ε.sub.0                                                I.sub.3 + ε.sub.3 + ε.sub.2 +                                 ε.sub.1 + ε.sub.0                  P.sub.4 + ε.sub.4                                                         I.sub.4 ' = P.sub.4 + ε.sub.4 + P.sub.3 + ε.sub.3 +           P.sub.2 + ε.sub.2 + P.sub.1 + ε.sub.1 + I.sub.0 +             ε.sub.0        I.sub.4 + ε.sub.4 + ε.sub.3 +                                 ε.sub.2 + ε.sub.1                                             + ε.sub.0                                  . . . etc.                                                                    __________________________________________________________________________

The recursive nature of the decoder is such that the picture is rebuiltfrom all preceding frames from the `I `start frame. This also applies tothe errors created by each `P ` frame causing a build up of errors inthe decoder. Hence it is necessary to a decoder in the encoder loop toprevent such a recursive build up.

However, the nature of the recursion is quite different for alogarithmic `B` coding system as described above. Assuming that theencoding operation is performed without the feedback loop, the encoderwould produce a sequence of frames lows for a 3-level temporaldecimation (8 frames): Frame Encoded Frame Decoded Frame

    ______________________________________                                        Start Frame Encoded Frame                                                                              Decoded Frame                                        ______________________________________                                        I.sub.0     I.sub.0      I.sub.0 + ε.sub.0                            I.sub.8     I.sub.8      I.sub.8 + ε.sub.8                            I.sub.4     B.sub.4 = I.sub.4 - (I.sub.0 + I.sub.8)/2                                                  B.sub.4 + ε.sub.4                            I.sub.2     B.sub.2 = I.sub.2 - (I.sub.0 + I.sub.4)/2                                                  B.sub.2 + ε.sub.2                            I.sub.6     B.sub.6 = I.sub.6 - (I.sub.4 + I.sub.8)/2                                                  B.sub.6 + ε.sub.6                            I.sub.1     B.sub.1 = I.sub.1 - (I.sub.0 + I.sub.2)/2                                                  B.sub.1 + ε.sub.1                            I.sub.3     B.sub.3 = I.sub.3 - (I.sub.2 + I.sub.4)/2                                                  B.sub.3 + ε.sub.3                            I.sub.5     B.sub.5 = I.sub.5 - (I.sub.4 + I.sub.6)/2                                                  B.sub.5 + ε.sub.5                            I.sub.7     B.sub.7 = I.sub.7 - (I.sub.6 + I.sub.8)/2                                                  B.sub.7 + ε.sub.7                            ______________________________________                                    

As stated previously, the order of `B ` frame generation is notimportant at the er since there is independence between the generationof each frame. But, the decoder must recreate the `I ` frames in thefollowing defined order:

    __________________________________________________________________________                                               Decoded Frame -                    Decoded Frame                              With Expanded Error                __________________________________________________________________________                                               Terms                              I.sub.0 '                                  = I.sub.0 + ε.sub.0        I.sub.8 '                                  = I.sub.8 + ε.sub.8        I.sub.4 ' = B.sub.4 + ε.sub.4 + (I.sub.0 ' + I.sub.8 ')/2                                                        = I.sub.4 + ε.sub.4 +                                                 ε.sub.0 /2                                                            + ε.sub.8 /2               I.sub.2 ' = B.sub.2 + ε.sub.2 + (I.sub.0 ' + I.sub.4 ')/2 =           I.sub.2 + ε.sub.2 + ε.sub.0 /2 + (ε.sub.4 +           ε.sub.0 /2 + ε.sub.8 /2)/2 = I.sub.2 + ε.sub.2 +                                                 3*ε.sub.0 /4 +                                                        ε.sub.4 /2                                                            + ε.sub.8 4                I.sub.6 ' = B.sub.6 + ε.sub.6 + (I.sub.4 ' + I.sub.8 ')/2 =           I.sub.6 + ε.sub.6 + (ε.sub.4 + ε.sub.0 /2 +           ε.sub.8 /2)/2 + ε.sub.8 /2 = I.sub.6 + ε.sub.6 +                                                 ε.sub.0 /4                                                            + ε.sub.4 /2 +                                                        3*ε.sub.8 /4               I.sub.1 ' = B.sub.1 + ε.sub.1 + (I.sub.0 ' + I.sub.2 ')/2 =           I.sub.1 + ε.sub.1 + ε.sub.0 /2 + (ε.sub.2 +           3*ε.sub.0 /4 + ε.sub.4 2 + ε.sub.8 /4)/2                                                         = I.sub.1 + ε.sub.1 +                                                 7*ε.sub.0 /8 +                                                        ε.sub.2 /2                                                            + ε.sub.4 /4 +                                                        ε.sub.8 /8                 I.sub.3 ' = B.sub.3 + ε.sub.3 + (I.sub.2 ' + I.sub.4 ')/2 =           I.sub.3 + ε.sub.3 + (ε.sub.2 + 3*ε.sub.0 /4 +         ε.sub.4 /2 + ε.sub.8 /4)/2 + (ε.sub.4                 + ε.sub.0 /2 + ε.sub.8 /2)/2                                                                             = I.sub.3 + ε.sub.3 +                                                 5*ε.sub.0 /8 +                                                        ε.sub.2 /2                                                            + 3*ε.sub.4 /4 +                                                      3*ε.sub.8 /8               I.sub.5 ' = B.sub.5 + ε.sub.5 + (I.sub.4 ' + I.sub.6 ')/2 =           I.sub.5 + ε.sub.5 + (ε.sub.4 + ε.sub.0 /2 +           ε.sub.8 /2)/2 + (ε.sub.6 + ε.sub.0 /4                 + ε.sub.4 /2 + 3*ε.sub.8 /4)/2                                                                           = I.sub.5 + ε.sub.5 +                                                 3*ε.sub.0 /8 +                                                        3*ε.sub.4 /4 +                                                        ε.sub.6 /2                                                            + 5*ε.sub.8 /8             I.sub.7 ' = B.sub.7 + ε.sub.7 + (I.sub.6 ' + I.sub.8 ')/2 =           I.sub.7 + ε.sub.7 + (ε.sub.6 + ε.sub.0 /4 +           ε.sub.4 /2 + 3*ε.sub.8 /4)/2 + ε.sub.8 /2                                                        = I.sub.7 + ε.sub.7 +                                                 ε.sub.0 /8                                                            + ε.sub.4 /4 +                                                        ε.sub.6 /2                                                            + 7*ε.sub.8 /8             __________________________________________________________________________

Now the error components are almost entirely noise and uncorrelated,hence the noise power for each frame is given as the sum of each errorsquared, i.e.:

    ______________________________________                                        Frame Noise Power                                                             ______________________________________                                        0:    E.sub.0.sup.2 = (ε.sub.0).sup.2                                 8:    E.sub.8.sup.2 = (ε.sub.8).sup.2                                 4:    E.sub.4.sup.2 = (ε.sub.4).sup.2 + (ε.sub.0 /2).sup.2           + (ε.sub.8 /2).sup.2                                           2:    E.sub.2.sup.2 = (ε.sub.2).sup.2 + (3*ε.sub.0                  /4).sup.2 + (ε.sub.4 /2).sup.2 + (ε.sub.8 /4).sup.2           2                                                                       6:    E.sub.6.sup.2 = (ε.sub.6).sup.2 + (ε.sub.0 /4).sup.2           + (ε.sub.4 /2).sup.2 + (3*ε.sub.8 /4).sup.2            1:    E.sub.1.sup.2 = (ε.sub.1).sup.2 + (7*ε.sub.0                  /8).sup.2 + (ε.sub.2 /2).sup.2 + (ε.sub.4 /4).sup.2           + (ε.sub.8 /8).sup.2                                            3:    E.sub.3.sup.2 = (ε.sub.3).sup.2 + (5*ε.sub.0                  /8).sup.2 + (ε.sub.2 /2).sup.2 + (3*ε.sub.4                   /4).sup.2 + (3*ε.sub.8 /8).sup.2                                5:    E.sub.5.sup.2 = (ε.sub.5).sup.2 + (3*ε.sub.0                  /8).sup.2 + (3*ε.sub.4 /4).sup.2 + (ε.sub.6                   /2).sup.2 + (5*ε.sub.8 /8).sup.2                                7:    E.sub.7.sup.2 = (ε.sub.7).sup.2 + (ε.sub.0 /8).sup.2           + (ε.sub.4 /4).sup.2 + (ε.sub.6 /2).sup.2 +                  (7*ε.sub.8 /8).sup.2                                            ______________________________________                                    

It is interesting to note that, unlike the `P ` prediction method, thesum of the noise components can be arranged to be a constant independentof frame number provided the noise components ε₄, ε₂, ε₆, ε₁, ε₃, ε₅, ε₇are adjusted through the quantization control in the following way:

For E₄ =E₀ =E₈ =ε,

    ε.sub.4 =√(ε.sup.2 -(ε/2).sup.2 -(ε/2).sup.2)=0.7071*ε(=√0.5*ε)

Now that E₄ =ε, and setting, E₂ =E₆ =ε gives

    ε.sub.2 =√(ε.sup.2 -(3*ε/4).sup.2 -((0.7071*ε)/2).sup.2 -(ε/4).sup.2)=0.5*ε

    ε.sub.6 =√(ε.sup.2 -(ε/4).sup.2 -((0.7071*ε)/2).sup.2 -(3*ε/4).sup.2)-0.5*ε

and setting E₁₌ E₃ =E₅ =E₇ =ε gives

    ε.sub.1 =√(ε.sup.2 -(7*ε/8).sup.2 ((0.5*ε)/2).sup.2 -(0.7071*ε/4).sup.2 -(ε/8))=0.3536*ε(=√0.125*ε)

    ε.sub.3 =√(ε.sup.2 -(5*ε/8).sup.2 -((0.5*ε)/2).sup.2 -(0.7071*3*ε/4).sup.2 -(3*ε/8))=0.3536*ε

    ε.sub.5 =√(ε.sub.2 -(3*ε/8).sup.2 -(0.7071*3*ε/4).sup.2 -((0.5*ε)/2).sup.2 -(5*ε/8))=0.3536*ε

    ε.sub.7 =√(ε.sup.2 -(ε/8).sup.2 -(0.7071*ε/4).sup.2 -((0.5*ε)/2).sup.2 -(7*ε/8))=0.3536*ε

There is clearly a relationship between the temporal decimation stageand the noise component such that a noise reduction by√(0.5) at eachadditional stage ensures a constant noise component in each frame.However, the requirement for reducing quantization noise for highertemporal frequency bands obviously reduces coding efficiency and thishas to be matched against the recursive noise addition of the closedloop encoder.

This system of noise weightings has been implemented and the noisevalues are close to that of the closed loop system for several systems.Reconstruction error measurements indicate <1 dB difference in systemnoise performance. Sometimes the noise in closed loop operation ishigher than open loop encoding, sometimes smaller. However, as notedmany times, the measurement of reconstruction error in a simple S/Nvalue can be misleading. Viewing tests conducted using both methodsclearly indicate that the errors of the type introduced by the open loopencoding process are far less objectionable than those from the closedloop system. The reason appears to be the nature of the noisecomponents. As already made clear, the closed loop encoder results innoise which is largely independent and persists for 1 frame only. Theopen loop encoder produces errors in various components, some of whichpersist for 1 frame only, whereas some fade in and out over severalframes. It seems that the eyes' temporal response is such thatflickering types of distortions are more readily visible than slowchanging errors. Anecdotal evidence shows up the effect as follows: whena picture is in still-frame mode some of the errors are not visibleuntil the sequence is played at normal speed; then these otherwiseinvisible `static ` errors become visible.

The characteristic of the noise components for both closed-loop andopen-loop encoding methods are shown in Table 1 below:

                                      TABLE 1                                     __________________________________________________________________________    Noise Power Components for the Open Loop Encoder                              Error                                                                              Scaling                                                                            F0  F1 F2 F3 F4 F5 F6 F7 F8                                         __________________________________________________________________________    ε.sub.0                                                                    1.0  1.0 0.766                                                                            0.562                                                                            0.391                                                                            0.25                                                                             0.141                                                                            0.062                                                                            0.016                                                                            0.0                                        ε.sub.1                                                                    0.125                                                                              0.0 0.125                                                                            0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.0                                        ε.sub.2                                                                    0.25 0.0 0.062                                                                            0.25                                                                             0.062                                                                            0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.0                                        ε.sub.3                                                                    0.125                                                                              0.0 0.0                                                                              0.0                                                                              0.125                                                                            0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.0                                        ε.sub.4                                                                    0.5  0.0 0.031                                                                            0.125                                                                            0.281                                                                            0.5                                                                              0.281                                                                            0.125                                                                            0.031                                                                            0.0                                        ε.sub.5                                                                    0.125                                                                              0.0 0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.125                                                                            0.0                                                                              0.0                                                                              0.0                                        ε.sub.6                                                                    0.25 0.0 0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.062                                                                            0.25                                                                             0.062                                                                            0.0                                        ε.sub.7                                                                    0.125                                                                              0.0 0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.0                                                                              0.125                                                                            0.0                                        ε.sub.8                                                                    1.0  0.0 0.016                                                                            0.062                                                                            0.141                                                                            0.25                                                                             0.391                                                                            0.562                                                                            0.766                                                                            1.0                                        Power                                                                              Sum: 1.0 1.0                                                                              1.0                                                                              1.0                                                                              1.0                                                                              1.0                                                                              1.0                                                                              1.0                                                                              1.0                                        __________________________________________________________________________

A pictorial representation of the noise components is shown in FIG. 4.

As can be seen from Table 1 and FIG. 4, the components due to errors inthe I frames ε₀ and ε₈, have the slowest fading errors, last 15 frames,and occupy a significant proportion of the overall error. Error ε₄ fadesin and out twice as fast extending over 7 frames. Errors ε₂ and ε₆ lastonly 3 frames each, and errors ε₁, ε₃, ε₅ and ε₇ only occur once duringtheir respective frames. The shorter duration errors have the lowestmagnitude.

The result is much more pleasing to the eye, and so is proposed for allapplications using `B ` frame decimation. Although the errordistribution is shown for 3-level temporal decimation of 8 frames,equivalent diagrams and underlying assumptions apply for any level oftemporal decimation; 1-level, 2-frame; 2-level, 4-frame; 3-level,8-frame (as shown); 4-level, 16-frame and so on.

FIG. 5 illustrates the encoder timing on a frame-by-frame basis.

FIG. 5 shows that the use of open-loop encoding considerably simplifiesthe system architecture and reduces the number of delay elementsrequired in both encoding and decoding. The diagram ignores therequirement for vector offsets used in motion vector estimation andcompensation, but since the timing is based on a per-frame basis, suchoffsets should not cause a problem. The normal delay of the encoder is5-frames, as can be seen from FIG. 5.

FIG. 6a illustrates the timing diagram of a decoder using a conventionaldecoding scheme. The normal delay of the decoder is 10 frames, thedecoder using 10 frames of memory which may be regarded as high.However, this can be reduced to 4 frames by adding an 8 frame delay tothe `I ` frames and reordering the `B ` frame sequence in the encoder asshown in FIG. 6b. Due to the removal of the feedback loop in theencoder, the `B ` frames can be generated in any order required.However, the decoder does require a particular order of operation torecover the original frames correctly. Namely, the I₄ frame must bedecoded before the I₂ and I₆ frames, and the I₂ and I₆ frames must bedecoded before the I₁, I₃, I₅, and I₇ frames. However, this is a simplerequirement which can easily be met.

Both encoder and decoder delays can be further reduced but at theexpense of inherent simplicity so this is not described.

The use of `B ` frame prediction is usually an improvement over `P `frames since the prediction error is reduced, classically by about 3dB.

The frames can be re-ordered in temporal significance (either literallyor symbolically) to the following order:

I₀ B₄ B₂ B₆ B₁ B₃ B₅ B₇ (I₈)

If the last 4 `B ` frames are deleted, then the sequence will play outwith an estimation of those frames, likewise, if frames B₄ and B₂ aredeleted. The method therefore allows a reasonable picture sequence to beplayed in the absence of the higher order `B ` frames. Naturally, if all`B ` frames are deleted, then the replayed sequence will be severelyperturbed, but with accurate motion vectors, the results can still besurprisingly good.

The method has been successfully used at data rates ranging from 20 Mbpsdown to 64 Kbps. However, so far, no direct comparison against the MPEGstructure has been made although it is believed that the open loopcoding method produces errors which are visually more acceptable.

The above-described method of temporally encoding video sequences hasbeen demonstrated to give very good results at a range of data rates fora range of applications. The use of `B ` frames allows the technique of`open-loop `encoding which produces similar reconstruction error values,but with much lower visibility leading to more pleasing picture quality.This is a significant development which is further enhanced by theknowledge that it is accomplished with a much simplified coding process.A conventional coder has to reconstruct the encoded signal to recoverthe signal which would be reconstructed in the receiver, whereas theopen loop encoder has no such requirement. The hardware saving in theencoder (excluding motion vector estimation and compensation) is likelyto be in the range of 30% to 50%.

The decoding of video sequences encoded with the logarithmic decimationprocess is necessarily longer than that for systems like MPEG1 usingonly forwards prediction with `P ` frames. However, it has been shownthat a decoder can be considerably simplified by the redistribution oftransmitted frames. This may be of lesser significance in a point topoint transmission system, but becomes important in point to multi-pointsystems where encoder complexity is less significant than decodercomplexity.

It should be noted that a `B ` frame system can form a subset of anothersystem using both `P `and `B `prediction methods as in MPEG2. Thereforesome of the `B` frame techniques herein described can be valid for asubset implementation.

Although illustrative embodiments of the invention have been describedin detail herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various changes and modifications can be effectedtherein by one skilled in the art without departing from the scope andspirit of the invention as defined by the appended claims.

I Claim:
 1. A motion compensated video processing method, comprising thesteps of:selecting a number of video frames including an I frame in avideo frame group comprised of plurality of I frames; and replacing said"I" frame by a "B" frame that is calculated as a difference between said"I" frame and the average of two neighboring "I" frames being located oneither side of said "I" frame.
 2. The method according to claim 1,further comprising applying weighting to predetermined frames of saidvideo frame group prior to selective summation.
 3. The method accordingto claim 2, further comprising decoding said video frame group byapplying weighting to said "B" frame prior to selective summation inorder to reconstruct the selected video frames.
 4. The method accordingto claim 3, wherein said decoding is of open-loop type, and saidweighting is applied by adjusting a quantization value for reducednoise.
 5. The method according to claim 1, further comprising decodingsaid video frame group by applying weighting to said "B" frame prior toselective summation in order to reconstruct the selected video frames.6. The method according to claim 5, wherein said decoding is ofopen-loop type, and said weighting is applied by adjusting aquantization value for reduced noise.